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A  Theory  for  Distributed  Signal  Detection  and  Data  Fusion 


Abstract 

This  research  attempts  to  develop  a  fundamental  understanding  of  the  issues  in¬ 
volved  in  the  design  and  performance  analysis  of  distributed  detection  schemes.  Such 
knowledge  is  currently  lacking.  This  is  especially  true  for  cases  with  statistically  depen¬ 
dent  observations  from  sensor  to  sensor,  a  practical  case  on  which  this  research  focuses. 
Some  emphasis  is  being  devoted  to  developing  design  algorithms  and  on  applications. 
The  goal  of  these  studies  is  to  produce  tools  and  techniques  for  pressing  practical  prob¬ 
lems.  We  classify  our  efforts  into  four  basic  areas:  properties  of  dependent  observations 
cases,  design  algorithms,  applications  and  image  fusion. 


1  Properies  of  Dependent  Observations  Cases 

Very  little  was  known  about  the  properties  of  optimum  distributed  detection  schemes  for 
cases  with  dependent  observations.  For  centralized  detection  cases,  likelihood  ratio  tests  are 
always  optimum.  This  suggested  an  interesting  question.  When  are  likelihood  ratio  tests 
optimum  at  distributed  sensors?  For  the  case  of  correlated  Gaussian  noise,  we  were  able 
to  obtain  conditions  under  which  this  must  occur  and  conditions  under  which  this  can  not 
occur  [1,  2].  NonGaussian  noise  cases  were  considered  in  a  later  series  of  papers  [3,  4,  5,  6], 
where  we  found  the  problem  is  considerably  more  complex,  however  the  optimum  detectors 
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follow  a  pattern  which  can  be  used  to  predict  the  general  forms  of  the  optimum  processing 
without  complex  computations. 

We  are  very  excited  about  the  latest  results  we  have  obtained  [7].  Given  a  distributed 
scheme  which  uses  a  given  number  of  bits  for  the  decisions  at  all  but  the  last  sensor,  we  can 
show  that  the  last  sensor  should  not  use  more  than  a  certain  number  of  bits  in  its  decision. 
If  more  bits  are  used  at  the  last  sensor,  then  the  same  performance  can  be  obtained  by  using 
less  bits  which  implies  lower  speed  communications.  The  results  in  [7]  are  extended  in  [8,  9] 
for  cases  with  more  than  two  hypothesis.  In  [8,  9],  we  further  show  that  in  some  common 
cases,  the  maximum  number  of  bits  that  should  be  used  at  each  sensor  is  much  less  the 
number  given  in  [7].  This  is  true  for  the  important  case  of  known  signals  in  uncorrelated 
Gaussian  noise.  It  is  even  true  in  many  cases  with  correlated  Gaussian  noise.  We  also 
investigate  how  a  finite  number  of  quantization  levels  should  be  allocated  across  the  sensors 
in  [8,  9]. 

2  Design  Algorithms 

The  Neyman  Pearson  criterion  has  been  a  subject  of  much  controversy  and  confusion  when 
studied  for  distributed  detection  cases.  For  example,  several  researchers  have  claimed  some 
results  produced  by  other  researchers  are  incorrect.  While  none  of  our  papers  were  involved 
in  this  controversy,  the  exchanges  of  the  other  researchers  motivated  us  to  attempt  to  uncover 
the  truth.  In  a  very  recent  series  of  papers  [10,  11],  we  have  finally  clarified  the  confusion 
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and  proved  some  Theorems  that  clearly  describe  the  true  situation.  The  Theorems  describe 
methods  for  finding  optimum  distributed  detection  systems.  The  proofs  draw  on  our  pre¬ 
vious  research  [12]  while  adding  some  critical  new  ideas,  including  new  efficient  numerical 
algorithms  for  finding  optimum  schemes.  In  a  second  series  of  papers  [13]  we  finally  show  that 
randomization  is  generally  necessary  to  achieve  optimum  performance  at  the  fusion  center. 
Further,  we  provide  the  first  correct  algorithm  for  finding  optimum  fusion  rules  employing 
randomization  [13].  A  general  approach  for  updating  the  fusion  rule  is  provided  that  is  also 
applicable  to  Bayesian  problems. 

New  efficient  algorithms  for  finding  Bayesian  optimum  distributed  decision  rules  for 
cases  with  dependent  observations  were  recently  given  in  [7].  These  algorithms  are  based 
on  a  new  discretized  Gauss-Seidel  approach.  The  convergence  of  these  algorithms  is  also 
proven  in  [7].  In  particular,  we  have  shown  that  these  algorithms  converge  in  a  finite  number 
of  iterations  and  that  as  the  discretization  step  sizes  approach  zero  the  solutions  approach 
person-by-person  optimum  solutions.  A  person-by-person  solution  is  one  that  can  not  be 
improved  by  changing  only  one  decision  rule  at  a  time. 

3  Applications 

Investigating  practical  applications  of  distributed  signal  detection  is  a  topics  to  which  we 
have  devoted  considerable  effort.  One  application  involves  fusing  the  data  obtained  from 
multiple  surveillance  systems  to  achieve  improved  performance.  Testbeds  have  been  con- 
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structed  at  a  number  of  DoD  laboratories.  Another  important  application  involves  the 
design  of  wireless  communication  receivers.  I  have  recently  obtained  some  analytical  re¬ 
sults  [14]  which  suggest  that  distributed  processing  techniques  can  reduce  the  complexity 
of  wireless  communication  receivers  without  any  significant  loss  in  performance.  Our  initial 
investigations  [14]  have  focused  on  frequency  shift  keying  modulation.  More  recently,  we 
have  considered  more  complicated  modulation  schemes,  including  frequency  hopped  spread 
spectrum  approaches  [15].  Our  latest  investigations  in  this  area  involve  considering  multiuser 
distributed  detection  problems  [16, 17].  We  appear  to  be  the  first  group  to  consider  this  topic 
which  appears  to  have  important  applications  in  ad-hoc  networks,  a  hot  new  research  area. 
Our  investigations  into  distributed  multiuser  detection  have  also  led  to  some  contributions 
to  centralized  multiuser  detection  theory  [18,  19,  20,  21,  22].  We  developed  a  new  multiuser 
detection  algorithm  [18,  19]  that  appears  to  perform  better  than  any  existing  suboptimum 
algorithms  with  complexity  which  grows  linearly  in  the  number  of  users.  In  fact,  in  a  large 
set  of  interesting  cases  its  performance  is  almost  indistinguishable  from  optimum.  We  have 
also  developed  [20,  21,  22]  some  highly  efficient  iterative  multiuser  detection  algorithms  for 
turbo  coded  CDMA  signals.  Our  algorithms  provide  excellent  performance  while  yielding  the 
lowest  possible  complexity  of  any  similar  algorithms  we  have  seen.  These  schemes  are  even 
applicable  to  nonGaussian  noise  cases  using  a  new  approach  we  developed  [23,  24,  22,  25,  26] 
which  uses  the  expectation  maximumization  algorithm  to  fit  the  parameters  of  a  Gaussian 
mixture  model. 


5 


Our  most  recent  work  has  focused  on  using  antenna  arrays  at  the  transmitter  also, 
originally  in  distributed  detection  based  systems.  In  particular  we  have  been  investigating 
space-time  coding  which  involves  coding  spatially  across  the  antenna  array  in  addition  to 
coding  across  time.  Space-time  coding  is  an  excellent  way  to  combine  diversity  gain  and 
coding  gain.  We  were  the  first  group  to  publish  a  journal  paper  suggesting  serially  concate¬ 
nated  space-time  codes  using  a  recursive  inner  code  to  achieve  significant  interleaver  gain 
[27,  28,  5].  A  highly  efficient  iterative  receiver  employing  a  Turbo  decoding  algorithm  was 
outlined  in  [27,  28,  5].  These  investigations  convinced  us  that  a  systematic  space-time  code 
design  procedure  was  lacking  so  we  developed  a  new  procedure  for  computing  and  bounding 
diversity  and  coding  gain  for  flat,  slow  fading  channels  in  [29,  30].  This  procedure  was  used 
to  find  new  optimum  codes  in  [31,  32].  The  performance  of  these  codes  was  compared  to  that 
for  popular  existing  codes  in  [31,  32].  These  comparisons  show  the  excellent  performance  of 
the  codes  in  [31,  32],  illustrating  the  utility  of  the  theory  in  [29,  30].  Our  most  recent  inves¬ 
tigations  on  this  topic  consider  multiple  trellis  coded  modulation  [33,  34].  An  efficient  design 
approach  for  both  slow  and  fast  fading  cases  is  proposed  in  [33,  34].  The  resulting  codes 
outperform  existing  space-time  codes.  The  combination  of  CDMA  with  space-time  coding 
is  investigated  in  [35,  36,  22].  The  main  contribution  is  a  very  low  complexity  receiver  that 
performs  the  decoding,  demodulation,  and  multiuser  detection  in  an  iterative  fashion  using 
a  successive  cancellation  approach. 
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4  Image  Fusion 


The  objective  of  image  fusion  is  to  combine  information  from  multiple  images  of  the  same 
scene.  The  result  of  image  fusion  is  a  single  image  which  is  more  suitable  for  human  and 
machine  perception  or  further  image  processing  tasks.  In  our  research  [37,  38,  39],  a  generic 
image  fusion  framework  based  on  multiscale  decomposition  is  studied.  This  framework  pro¬ 
vides  freedom  to  choose  different  multiscale  decomposition  methods  and  different  fusion 
rules.  The  framework  includes  all  of  the  existing  multiscale-decomposition-based  fusion  ap¬ 
proaches  we  found  in  the  literature  which  did  not  assume  a  statistical  model  for  the  source 
images.  Different  image  fusion  approaches  are  investigated  based  on  this  framework.  Some 
evaluation  measures  are  suggested  and  applied  to  compare  the  performance  of  these  fusion 
schemes.  The  majority  of  the  research  focuses  on  fusing  same  sensor  images.  In  partic¬ 
ular,  visual  images  are  used  in  the  majority  of  the  tests.  The  comparisons  indicate  that 
our  framework  includes  some  new  approaches  which  outperform  the  existing  approaches  for 
the  cases  we  consider.  The  topic  of  image  fusion  has  also  been  studied  for  an  interesting 
new  camera,  the  omni  camera  in  [40].  At  a  more  basic  level,  we  have  studied  algorithms  to 
measure  fusion  quality  [41,  39].  These  algorithms  are  suitable  for  many  applications,  but 
they  are  particularly  well  suited  for  image  fusion.  A  study  of  new  registration  methods  for 
image  fusion  was  also  undertaken  [39]. 
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5  Contributions 


The  project  has  made  significant  contributions  to  the  field  of  Electrical  Engineering  and  in 
particular  to  the  study  of  distributed  signal  processing  systems.  The  main  contributions 
have  been  towards  developing  the  fundamental  theory  of  this  topic,  which  has  been  lacking. 
Such  studies  should  stimulate  future  breakthroughs.  Although  we  did  realize  this  theory 
was  not  in  place,  even  we  were  amazed  at  the  basic  nature  of  some  of  our  results.  It  was 
surprising  that  results  of  this  type  were  new.  This  illustrated  to  us  how  little  is  really  known 
about  the  underlying  theory  of  distributed  signal  processing.  For  example,  prior  to  our 
work  it  was  not  known  that  the  number  of  sensor  decision  bits  used  at  one  sensor  can  limit 
the  number  that  should  be  used  at  another  sensor.  This  is  a  very  basic  property  that  relies 
heavily  on  the  nonuniqueness  of  the  overall  decision  rule  realized  by  a  particular  set  of  sensor 
rules  and  a  fusion  rule.  Although  the  result  makes  sense  to  us  now,  even  we  were  surprised 
by  this  finding.  We  were  also  surprised  by  the  number  of  errors  and  misconceptions  in  the 
existing  theory  for  Neyman-Pearson  optimum  distributed  signal  detection.  We  believe  that 
our  work  on  this  topic  will  lead  to  numerous  new  developments  and  theory.  Prior  to  this  the 
theory  of  Neyman-Pearson  optimum  distributed  schemes  was  thought  to  be  extremely  hard 
to  develop  for  dependent  observation  cases.  Our  recent  results  indicate  this  theory  may  not 
be  so  difficult  to  develop. 

Since  our  results  are  at  such  a  basic  level,  we  believe  they  have  contributed  to  many 
disciplines  and  in  particular  to  the  broad  topic  of  data  fusion  which  spans  many  areas  of 
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science  and  engineering.  We  outline  here  the  areas  we  expect  will  benefit  from  this  research. 
The  theory  of  distributed  decision  making  is  important  in  many  areas  outside  Electrical 
Engineering.  It  has  applications  in  any  area  involving  team  decision  making  by  groups  of 
smart  machines  or  humans.  Particular  application  areas  include  financial  institutions,  air- 
traffic  control,  oil  exploration,  medical  diagnosis,  military  command  and  control,  electric 
power  networks,  weather  prediction,  manufacturing,  computer-based  navigation  systems, 
law  enforcement,  robotics,  and  industrial  organizations.  Our  basic  theory  on  how  to  design 
distributed  decision  making  systems  should  influence  developments  in  these  application  areas. 
We  should  mention  that  our  topic  is  closely  related  to  an  important  type  of  data  fusion,  called 
decision  fusion,  that  has  its  roots  in  robotics,  artificial  intelligence,  mathematics,  information 
theory  and  computing.  We  have  been  interfacing  with  these  communities  through  the  Fusion 
conferences  (Fusion  98  and  Fusion  99)  which  have  been  held  the  last  few  years.  We  have 
been  active  in  running  this  conference  and  in  organizing  a  multidisciplinary  Data  Fusion 
Society. 

We  have  trained  PhD  level  researchers  for  the  signal  processing,  communications 
and  computing  industries,  areas  where  highly  trained  personnel  are  desperately  needed. 
Wherever  possible,  we  have  encouraged  these  researchers  to  consider  faculty  positions.  In 
cases  where  interest  exists  the  student  has  been  trained  and  mentored  specifically  for  this 
purpose  since  we  believe  developing  highly  qualified  faculty  members  is  an  important  role 
for  a  University.  It  is  also  very  important  for  the  future  success  of  our  nation.  Thus  far  four 
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PhD  students  have  graduated  who  were  at  least  partially  supported  by  this  research  grant 
(or  by  Lehigh’s  matching  funds).  Two  more  PhD  students  who  were  partially  supported  by 
this  research  grant  will  grauate  next  year. 

Clearly  we  are  proud  of  our  strong  theoretical  contributions,  but  we  are  also  proud 
of  the  more  practical  contributions  we  have  made.  We  have  developed  practical  design  algo¬ 
rithms  for  distributed  signal  detection  systems  and  we  have  demonstrated  the  applications 
of  our  theory  to  wireless  communication  receivers.  We  are  aware  of  several  experimental 
surveillance  systems  constructed  at  several  laboratories.  The  designers  have  struggled  due 
to  a  lack  of  theory  for  the  design  of  these  systems.  Our  results  should  be  helpful  in  this 
regard.  In  fact,  we  have  also  convinced  personnel  at  Lucent  Technologies  [15]  that  our  ideas 
could  be  applied  in  the  implementation  of  practical  communication  receivers. 
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